System and method for imaging of curved surfaces

ABSTRACT

A system for capturing a composite image of an object with a curved surface includes a conveyor configured to transport the object to be imaged to a predetermined imaging position. A sensor is configured to produce a signal when the object to be imaged is at the predetermined position, and several cameras are arranged to photograph the object at the predetermined position from a plurality of different angles. A tracking module is used to receive the signal from the sensor, and output an actuating signal to the several cameras, such that each camera captures an image when the actuating signal is received. A processing device receives a captured image from each of the several cameras, manipulates the received images, and generates a composite image based on the manipulated images.

PRIORITY CLAIM AND REFERENCE TO RELATED APPLICATION

The application claims priority under 35 U.S.C. §119 from priorprovisional application Ser. No. 61/052,154, which was filed May 9,2008.

FIELD OF THE INVENTION

A field of the invention is visioning. An example application of theinvention includes imaging of curved object surfaces.

BACKGROUND OF THE INVENTION

In production industries, a product is generally inspected to ensurequality before it is distributed. Traditionally, such inspection isperformed by a human, but it has been found that humans tend to have ahigh degree of failure due to distraction, illness and/or othercircumstances. For this reason, many industries turn to machine visionto perform inspections of production goods. Machine vision is a usefulalternative to human inspection when high speed, high magnification,24-hour operation, and/or repeatable measurements are required.

As an example, wine makers use machine vision to inspect empty bottlesfor chips, cracks or other imperfections, as well as dirt and dust. Theyalso use machine vision to verify the fill level and cork positioning offilled wine bottles, and to approve full cases of wine before they arereleased for distribution. Additionally, machine vision can be used tocheck the labels on wine bottles for both presence and placement.

The California wine industry produced 2.7 billion bottles of wine in2005. Wineries' bottling lines currently produce up to 300 bottles perminute. Accordingly, it is necessary for winemakers to have aninspection system that not only successfully detects and rejectssubstandard product, but does so quickly and efficiently. Put anotherway, a machine vision imaging system used in any industry must not onlyidentify defects, but must also keep up with demand placed on the systemby the manufacturing line.

Conventional machine vision systems rely on mechanical stops or anorientation device to position and orient the product to be inspected sothat the product is in a predetermined position. Orienting the productso that the positioning is known allows for a reduction in requiredprocessing power, but the time required to mechanically orient eachproduct to be inspected imposes limitations on the potential throughputof the inspection system.

SUMMARY OF THE INVENTION

One embodiment of the invention is a system and method for capturing acomposite image of an object with a curved surface. The inventionincludes a conveyor configured to transport the object to be imaged to apredetermined imaging position. A sensor produces a signal when theobject to be imaged is at the predetermined position, and severalcameras are arranged to photograph the object at the predeterminedposition from a plurality of different angles. A tracking module is usedto receive the signal from the sensor, and output an actuating signal tothe several cameras, such that each camera captures an image when theactuating signal is received. A processing device receives a capturedimage from each of the several cameras, manipulates the received images,and generates a composite image based on the manipulated images.

In another embodiment, a vision imaging system is provided, the systemincluding multiple or cameras configured to capture an image of apredetermined area. A sensor is connected to the cameras, so that thesensor transmits a signal to the cameras when an object to be imaged iswithin the predetermined area. An image processing device is configuredto receive an image from each of the multiple cameras, and perform animage manipulation process on each of said received images, based on apredetermined object shape. The image processing device then generates acomposite image of the object based on each of the manipulated images.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system according to an embodiment of theinvention;

FIG. 2 is a screenshot of a program used to operate the system shown inFIG. 1;

FIG. 3 is a flowchart illustrating a method of imaging an object with acurved surface using the system of FIG. 1; and

FIG. 4 is a composite image produced by the system of FIG. 1.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention relates to a machine vision system that captures acomposite image of an object with a curved surface. A processing devicereceives a captured image from each of the several cameras, manipulatesthe received images, and generates a 360° composite image based on themanipulated images. The composite image generated by the system can beused to inspect the product of a manufacturing or production line at arelatively high speed. For example, the system is capable of inspectingin excess of 200 objects per minute, and the speed of inspection couldbe increased to accommodate approximately 1200 objects per minute,depending on bottle size, inspections and available computertechnologies.

Referring now to FIG. 1, an embodiment of a machine vision imagingsystem 10 is shown. The system 10 includes a conveyor 12 that is used tocarry an object to be inspected through the system. Further included inthe system 10 is a sensor 14 which is in communication with a trackingmodule 16. The tracking module 16 is, in turn, connected to each of twoor more cameras 18 provided within the system 10, and each of thecameras is connected to an image processing apparatus 20. To helpprevent ambient light from entering the system 10 and to protect thevarious components, an opaque housing 22 covers the entire system 10.Alternatively, the housing 22 may cover all portions of the system 10except the image processing apparatus 20.

Within the housing 22, the system 10 is preferably lit using high-outputlamps (not shown) mounted directly over the conveyor 12. This mountingposition helps to minimize extraneous shadows. The mounting also createsa uniform light source within the system 10, which makes it easier toproduce repeatable measurements. The high-output lamps are, for example,36-120 watt high-output fluorescent bulbs, which have a very high lightproduction/volume ratio. However, other high-output light sources, suchas LED lighting, are also contemplated.

The conveyor 12 is a device that is used to carry objects to beinspected through the machine vision imaging system 10. For example, amoving belt or a series of rollers may be used. The conveyor bringsobjects from a production line into the system 10 and to a predeterminedimaging and inspection area 26, and carries the objects from the imagingarea out of the system to continue along the production line.

The sensor 14 is located at or near the imaging area 26, and is used toindicate that an object to be inspected has entered the inspection area.Sensor 14 is, for example, a photosensor that includes a light emittingportion 28 and a light sensitive portion 30. The light emitting portion28 is configured to emit a beam of coherent light that crosses conveyor12 and contacts the light sensitive portion 30. The sensor 14 producesno output as long as the beam of light emitted by the light emittingportion 28 remains in contact with the light sensitive portion 30.However, if the beam of light is broken or otherwise prevented fromcontacting the light sensitive portion 30, the sensor 14 produces atrigger signal that is output to the tracking module 16. While thetrigger signal may be any measurable voltage, it is preferred that thesignal be a relatively high voltage, such as a 24 volt signal, whichhelps to provide a more noise-immune signal. Also, even though aphotosensor is described above, it is contemplated that sensor 14 mayalternatively include other types of sensor, such as a pressure sensoror the like.

The trigger signal output by the sensor 14 is provided as an input tothe tracking module 16. Tracking module 16 is a controller that is usedto track the objects that are inspected by the system 10, andparticularly to identify objects that do not pass inspection. Inaddition to tracking objects that do not pass inspection, trackingmodule 16 also acts as a fail-safe mechanism to help prevent substandardor uninspected objects from passing through the system 10. The trackingmodule 16 also provides as an output, a camera activation signal. Thecamera activation signal is output to each of the cameras 18 included inthe system 10, and is generated upon receipt of the trigger signal fromthe sensor 14.

The magnitude of the camera activation signal output by the trackingmodule 16 is determined by the magnitude of the signal accepted at aninput port 32 of each of the cameras 18. Accordingly, the cameras 18preferably include an input port 32 that accepts a relativelyhigh-voltage (e.g., approximately 12 to 24 volts) activation signal. Asdiscussed above, the high voltage improves the noise-immunity of thesystem. Each camera 18 is preferably a digital still-frame cameracapable of capturing images at a resolution of 640×480 pixels or better.Alternatively, it is contemplated that high-speed digital video camerascould be used in place of digital still-frame cameras. The high-outputlights allow the cameras 18 to operate using a short shutter speed ofabout 100 μs to 1 ms depending on conveyor speed, and an aperture of F4to F22 depending on the desired depth of field. These settings allow fora large depth of field, which aids in keeping the curved surface of theobject in focus as it turns away from the camera 18. The cameras 18 arepreferably arranged so that the object in the inspection area 26 isphotographed from all angles. Additionally, there should be some overlapin the viewing angle of the cameras, to aid in image manipulationperformed by the image processing apparatus 20. Each camera 18 should besecurely mounted, such that the position of each camera is fixed andknown. For example, in the system 10 shown in FIG. 1, there are fourcameras 18 arranged around an inspection area 26 at 90 degree intervals,and each camera has a 120° field of view.

Each of the cameras 18 also includes an output port 34 that outputs acaptured image to the image processing apparatus 20. The output port 34preferably transmits the captured image according to the gigabitEthernet standard (i.e., the standard defined by IEEE 802.3-2005), whichincreases the possible distance between the camera 18 and the imageprocessing apparatus 20 without using repeaters or hubs. Alternatively,it is contemplated that any wired or wireless communication standard maybe used to transfer data from camera 18 to the image processingapparatus 20, provided that the data throughput rate is sufficient toexceed the demands of the system 10.

Mounted substantially opposite (i.e., directly across viewing area 26from) each camera 18 is one or more light guides 36. The light guides 36are, for example, a passive light-gathering element made from a materialsuch as a fluorescing acrylic or the like that will take in lightproduced by the high-output lamps from above and/or below the conveyor12 and project the light. Alternatively, it is contemplated that each ofthe light guides 36 may include one or more light sources, such as lightemitting diodes or the like. The guides 36 are positioned opposite thecamera 18, and project light lines substantially parallel to theconveyor and in the direction of the camera to provide a high-contrastbackground for an image captured by the camera. The background formed bythe light guides 36 aids in identification of the object position, sizeand tilt in a captured image.

The image processing apparatus 20 receives images output from thecameras 18 via an input port 38. The apparatus includes a memory 40 anda processor 42, and a monitor, such as a touch-screen display device(not shown) for both receiving input from a user and providing output tothe user. To manage the demands of a high-throughput production line, itis important that the memory 40 is large enough to contain at least thecaptured images and a lookup table 44, and that the processor 42 iscapable of managing all tasks required. For example, an image processingapparatus may include 2 GB or more of system memory, and a multi-coreprocessor. Specifically, an example system may contain one or two Intel3.2 GHz Quad Core Extreme processors, depending on line rate, an IntelPro/1000 ethernet card, and 4 GB of system memory.

Images received via the input port 38 are stored into the memory 40, andonce the images are held in memory, the image processing apparatus 20relies on the lines projected by the light guides 36 in the backgroundof the captured images to help determine the location, size and tilt ofthe object in each image. To determine the location, size and tilt ofthe object in the captured image, the image processing apparatus locatesbright to dark transitions in the captured image along the lineprojected by the light guides. These transitions indicate the positionin the image where an edge of the object meets the light guide. Once theedges of the object are located, the relative size of the object in thecaptured image can be determined and compared to size information storedin the lookup table 44. Additionally, the edges can be used to indicatewhether or not the object is tilted, and by what amount. Accordingly, itis important that the light guide appear brighter than the object and/orlabel, so that the bright to dark transitions can be found.

Once the location, size, and tilt of the object is determined, eachobject can be “unwrapped,” creating a two-dimensional projection of thethree-dimensional object and correcting image distortion caused by thecurved surface of the object so that the surface of the object appearsto have been flattened. Finally, all of the unwrapped images arecombined into a single composite image, said to be captured by a“virtual camera” and referred to as a virtual camera image, which showsthe photographed object from all sides, similar to a linescan image.Based on the virtual camera image, the image processing apparatus 20 canbe used to verify various properties and characteristics of the objectcan be inspected, including presence and placement of labels,positioning of closures, and the like.

The image processing apparatus 20 further includes operating software 50for managing the image processing. FIG. 2 is a screenshot of theoperating software 50, as displayed by the display device. When runningthe software 50, the screen is divided into an image display portion 52,a statistical display portion 54, and a control portion 56. The imagedisplay portion 52 includes a camera image 58, and image controls 60allowing a user to zoom in or out on the displayed image 58, pan acrossthe image, both horizontally and vertically, and also to store thedisplayed image.

Statistical display portion 54 displays information regarding one of theplurality of cameras 18, or the composite image “virtual camera.” Theuser selects a camera from the camera selection control 62, and a table64 including various inspection tests that may be performed is displayedtogether with the results of selected tests. Finally, the controlportion 56 includes controls allowing the user to perform functions suchas display live video for a selected camera, enter a training mode,display camera settings, disable the camera, and the like.

By way of an example, the operation of the system 10 represented in FIG.1 will be described as used in a winery. That is, the objects beinginspected by the system 10 are wine bottles. However, it will beappreciated by those skilled in the art that the system 10 may be usedin any number of fields where rapid inspection of product is desired.Additionally, as shown in FIG. 1, four cameras are used to captureimages of the wine bottles.

FIG. 3 is a flowchart showing the method of operation. At the start ofthe operation, the conveyor 12 transports a bottle to the predeterminedinspection area 26 located within the system 10 (Block 70). The sensor14 senses when the bottle travelling on the conveyor 12 has reached thepredetermined inspection area 26. This can be done, for example, byaiming the light beam of a photosensor across the conveyor 12 at thepredetermined area 26. Then, when the beam is broken by the wine bottle,the bottle is in position for inspection. At that time, the sensor 14outputs a high-voltage trigger signal (Block 72).

When the trigger signal is received by the tracking module 16, themodule outputs an actuating signal to the input ports 32 of each of thefour cameras 18 (Block 74). The actuating signal is preferablyrelatively high voltage to improve the noise immunity of the system 10.

In response to the received signal, each camera 18 captures an image ofthe wine bottle. The captured images are then output to the imageprocessing apparatus 20 via camera output ports 34 (Block 76). Thecameras 18 are arranged to capture images of the wine bottle from allsides, and are mounted so that the camera position will not shift.Additionally, the arrangement of the cameras provides some overlap inthe field of view of each camera to aid in image processing. Forexample, the four cameras 18 provided in the system 10 are arranged in aring surrounding the predetermined inspection area 26, at 90 degreeintervals. The field of view for each camera 18 is approximately 120degrees. Thus, the field of view of each camera 18 overlaps with that ofthe adjacent cameras.

The image processing apparatus 20 receives the captured images andstores them in the system memory 40 (Block 78). The image processingapparatus 20 then determines the location, size and tilt of the bottlein each of the received images (Block 80). This process is performedusing a known edge-measurement algorithm, such as the edge-measurementalgorithm included in the Matrox Imaging Library. To determine thelocation, size and tilt of the bottle, the image processing apparatus 20first locates the light guides 36 in the captured image. The imageprocessing apparatus 20 then follows the light guide horizontally untila bright to dark transition is found. The image processing apparatus 20uses this transition to determine the position of the bottle edge.Accordingly, it is important that the light guides are brighter than thebottle and the label. Next, the image processing apparatus determineswhether or not the edge-finding algorithm successfully identified thelocation and tilt of the bottle (Block 82). If the edge-findingalgorithm is unable to determine the location and tilt of the bottle,the bottle is rejected (Block 92).

After determining the location and tilt of the bottle, each image of thebottle is “unwrapped” (Block 84), creating a 2-dimensional projection ofthe three-dimensional object and correcting image distortion that occursbecause of the curved surface of the bottle so that the surface of thebottle appears to have been flattened. The unwrapping process relies ona known warping algorithm, such as the algorithm provided in the MatroxImaging Library, used in conjunction with the lookup table 44 maintainedin memory 40 that stores information regarding the shape of the bottle.The use of the lookup table 44 helps particularly when correcting forposition and/or tilt, and/or size of the bottle.

Once the bottle has been identified and the images have been unwrappedor flattened, the flattened images are stitched together to form asingle composite image that shows the bottle from all sides (Block 86).FIG. 4 shows an example composite image for a wine bottle. Once created,the composite image is preferably displayed on the display device. Thedisplay may be used by an operator as an impetus to create newinspection routines, and may also help an operator troubleshoot thesystem in the event that a persistent problem with, for example, labelplacement is detected.

The composite image produced by the image processing apparatus 20 isinspected to verify object properties (Block 88). For example, the labelposition of the wine bottle may be verified. If the label is level, thenthe y-coordinates of the top edge of the label will be the same acrossthe entire image. Similarly, the edge-to edge distance between twolabels can be measured. Finally, various pattern matching algorithms canbe used to verify that the label that is applied to the bottle iscorrect. The measurements are then compared to predetermined standards(Block 90). If the measured properties fail to meet predeterminedstandards, then the image processing apparatus rejects the bottle (Block92). The process then returns to Block 70 to be repeated for the nextbottle.

Alternatively, if the bottle meets or exceeds the predeterminedstandards, the bottle is accepted (Block 94). At this point, the entireprocess returns to Block 70 to be repeated for the next bottle. As notedpreviously, the system 10 is capable of inspecting in excess of 200bottles per minute, up to approximately 1200 bottles per minute,depending on bottle size, inspections and available computertechnologies.

While specific embodiments of the present invention have been shown anddescribed, it should be understood that other modifications,substitutions and alternatives are apparent to one of ordinary skill inthe art. Such modifications, substitutions and alternatives can be madewithout departing from the spirit and scope of the invention, whichshould be determined from the appended claims.

Various features of the invention are set forth in the appended claims.

1. A system for capturing a composite image of an object with a curvedsurface, the system comprising: a conveyor configured to transport theobject to be imaged to a predetermined imaging position; a sensorconfigured to produce a signal when the object to be imaged is at saidpredetermined imaging position; a plurality of cameras configured tophotograph the object at said predetermined imaging position from aplurality of different angles; a tracking module configured to receivesaid signal from said sensor and output an actuating signal to saidplurality of cameras, wherein each camera captures an image when saidactuating signal is received; and a processing device that receives acaptured image from each of said plurality of cameras, manipulates thereceived images, and generates a composite image based on saidmanipulated images.
 2. The system of claim 1 further comprising one ormore light guides corresponding to each of said plurality of cameras,each said light guide positioned directly opposite said correspondingcamera.
 3. The system of claim 2, comprising four cameras and four pairsof light guides.
 4. The system of claim 1, wherein said plurality ofcameras are arranged in a ring surrounding said predetermined imagingposition.
 5. The system of claim 1, wherein said processing device usesan algorithm to correct the received images for distortion caused by thecurved surface of the object.
 6. The system of claim 5, wherein saidprocessing device judges whether or not one or more labels placed on theobject are positioned correctly based on said composite image.
 7. Thesystem of claim 6, wherein said image processing device is configured tojudge whether or not an object is defective based on a judgment resultof said label placement judgment.
 8. The system of claim 1, wherein saidsensor comprises a light beam that shines across said conveyor tocontact a photosensitive material.
 9. The method of claim 1, whereinsaid composite image represents a 360° image of the object.
 10. A methodfor creating a composite image of an object with a curved surfacecomprising the steps of: transporting the object to be imaged on aconveyor to a predetermined position; sending a signal to a trackingmodule when a sensor detects that the object to be imaged is at saidpredetermined position; outputting an actuating signal from a trackingmodule to a plurality of cameras upon receipt of said signal from thesensor; capturing an image of the object using each of the plurality ofcameras upon receipt of said actuating signal; and generating acomposite image from a processing device upon receipt of said capturedimages from each of the plurality of cameras.
 11. The method of claim10, wherein the processing device performs an image manipulation on eachof said received images, and said composite image is based on saidmanipulated images.
 12. The method of claim 10, further comprising thestep of analyzing said composite image to determine if one or morelabels placed on the object are positioned correctly.
 13. The method ofclaim 12, further comprising the step of rejecting said object if one ormore of said labels are not positioned correctly.
 14. The method ofclaim 10, wherein said composite image represents a 360° image of theobject.
 15. A vision imaging system comprising: a plurality of camerasconfigured to capture an image of a predetermined area; a sensorconnected to the cameras, wherein the sensor transmits a signal to thecameras when an object to be imaged is within said predetermined area;and an image processing device configured to receive an image from eachof said plurality of cameras, perform an image manipulation process oneach of said received images, and generate a composite image based oneach of said manipulated images; wherein said image processing devicemanipulates said received images according to a predetermined objectshape.
 16. The vision imaging system according to claim 15, wherein saidpredetermined object shape is stored in a look-up table.
 17. The visionimaging system of claim 15, further comprising a tracking moduleconfigured to track said object.
 18. The vision imaging system of claim17, wherein said image processing device judges whether one or morelabels placed on said object are correctly positioned using saidgenerated composite image.
 19. The vision imaging system of claim 18,wherein said tracking module is used to track objects that are judged tohave incorrectly positioned labels.
 20. The system of claim 15 furthercomprising one or more light guides corresponding to each of saidplurality of cameras, each said light guide positioned directly oppositesaid corresponding camera.